Fix /llm_log endpoint: replace brace-counting parser with one-JSON-per-line format#149
Conversation
Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/3e021d3b-8857-4593-a659-1eae83bff8f4 Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…f sensitive information' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates the /llm_log endpoint to parse logs as one JSON object per line, eliminating corruption caused by the previous brace-counting reconstruction approach.
Changes:
- Emit compact single-line JSON for LLM request/response/error logs.
- Simplify
/llm_logparsing tojson.loads()each non-empty line and return the last 10 entries. - Add integration tests covering multi-entry parsing, braces inside JSON strings, and the 10-entry cap.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
novelforge/llm/client.py |
Switch LLM log writes to single-line JSON (and redact Authorization) so /llm_log can parse reliably. |
novelforge/__init__.py |
Replace brace-counting parser with line-by-line JSON parsing and return last 10 entries. |
tests/test_integration.py |
Add tests validating the new one-JSON-per-line parsing behavior and truncation logic. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…isolation Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/f92c009b-bb39-4305-8e31-324b1a695711 Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>
|
Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/147f948c-2221-4cd9-860c-498288b76c8d Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The
/llm_logendpoint reconstructed multi-line indented JSON using{/}counting, which silently corrupts entries containing brace characters in string values or escaped quotes.Changes
novelforge/llm/client.py— Dropindent=2from the threellm_logger.info(json.dumps(...))calls (error, request, response). Each log entry is now a single compact line. Authorization headers are redacted before logging.novelforge/llm/image.py— Dropindent=2from the image request log write so allllm_loggerwriters emit single-line JSON consistently.novelforge/__init__.py— Replace the brace-counting loop with a plain JSONL line-by-line parser: each non-empty line is parsed withjson.loads(stripped), malformed lines are skipped, and acollections.deque(maxlen=10)keeps memory bounded at O(10) entries regardless of log file size. No buffering or resync logic is needed since all writers now emit single-line JSON.tests/test_integration.py— Add three tests toTestLLMLogusing a fully isolated_isolated_appfixture (patches all dirs:NOVELS_DIR,LOGS_DIR,SESSION_FILE_DIR,EXPORT_DIR) covering: correct parsing of multi-entry logs, entries with{/}inside string values, and the 10-entry truncation cap.